Beyond Maximum Likelihood: from Theory to Practice

نویسندگان

  • Jiantao Jiao
  • Kartik Venkat
  • Yanjun Han
  • Tsachy Weissman
چکیده

Maximum likelihood is the most widely used statistical estimation technique. Recent work by Jiao, Venkat, Han, and Weissman [1] introduced a general methodology for the construction of estimators for functionals in parametric models, and demonstrated improvements both in theory and in practice over the maximum likelihood estimator (MLE), particularly in high dimensional scenarios involving parameter dimension comparable to or larger than the number of samples. This approach to estimation, building on results from approximation theory, is shown to yield minimax rate-optimal estimators for a wide class of functionals, implementable with modest computational requirements. In a nutshell, a message of this recent work is that, for a wide class of functionals, the performance of these essentially optimal estimators with n samples is comparable to that of the MLE with n lnn samples. In the present paper, we highlight the applicability of the aforementioned methodology to statistical problems beyond functional estimation, and show that it can yield substantial gains. For example, we demonstrate that for learning tree-structured graphical models, our approach achieves a significant reduction of the required data size compared with the classical Chow–Liu algorithm, which is an implementation of the MLE, to achieve the same accuracy. The key step in improving the Chow–Liu algorithm is to replace the empirical mutual information with the estimator for mutual information proposed in [1]. Further, applying the same replacement approach to classical Bayesian network classification, the resulting classifiers uniformly outperform the previous classifiers on 26 widely used datasets.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Modified Maximum Likelihood Estimation in First-Order Autoregressive Moving Average Models with some Non-Normal Residuals

When modeling time series data using autoregressive-moving average processes, it is a common practice to presume that the residuals are normally distributed. However, sometimes we encounter non-normal residuals and asymmetry of data marginal distribution. Despite widespread use of pure autoregressive processes for modeling non-normal time series, the autoregressive-moving average models have le...

متن کامل

Maximum Likelihood Estimation of Parameters in Generalized Functional Linear Model

Sometimes, in practice, data are a function of another variable, which is called functional data. If the scalar response variable is categorical or discrete, and the covariates are functional, then a generalized functional linear model is used to analyze this type of data. In this paper, a truncated generalized functional linear model is studied and a maximum likelihood approach is used to esti...

متن کامل

Algebraic Statistics: Theory and Practice

Max Buot The Maximum Likelihood Degree of the Cauchy Location Likelihood. Maximum likelihood theory ensures that exactly one root of the Cauchy location likelihood equation is near 0, with all other roots bounded away from 0, in probability, as the sample size increases. However, the theory does not yield information concerning the total number of roots. In this talk, we determine the maximum l...

متن کامل

Beyond first order logic: From number of structures to structure of numbers: Part II

We study the history and recent developments in nonelementarymodel theory focusing on the framework of abstractelementary classes. We discuss the role of syntax and semanticsand the motivation to generalize first order model theory to nonelementaryframeworks and illuminate the study with concrete examplesof classes of models. This second part continues to study the question of catecoricitytrans...

متن کامل

مقایسه روش‌های طبقه‌بندی‌کننده حداکثر مشابهت و حداقل فاصله از میانگین در تهیه نقشه پوشش اراضی (مطالعه موردی: استان اصفهان)

Land cover maps derived from satellite images play a key role in regional and national land cover assessments. In order to compare maximum likelihood and minimum distance to mean classifiers, LISS-III images from IRS-P6 satellite were acquired in August 2008 from the western part of Isfahan. First, the LISS-III image was georeferenced. The Root Mean Square error of less than one pixel was the r...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1409.7458  شماره 

صفحات  -

تاریخ انتشار 2014